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^ — ^ . Abstract 

Literature in which the cloze procedure has heen used 
as a meastirement of listening oomprehensl^>n is reviewed 
In this paper. Secondly, suggestions for further research 
are offered. 



Aural Cloze: A Review of Literature 



-Wilson Taylor is credited with developing the cloze ' 
procedure, which he first presented at an Association for 
Education in Journalism convention workship in 1953. Soon 
after, "a description of the method appeared in the Joumftlisa. 

Quart^ly (Taylor, 1953). 

Originally, cloze was intended as a new approach to 
"readability." It derived, its name from the term "closure" 
which refers to the tendency of human heings to complete a 
pattern which is familiar, but not-quite-finished. In 
language, "people will complete an Incomplete sentence 

pattern, "It is raining and dogs." Most pecple, 

reading or hearing this sentence, would fill the blatik In 
with the word "cats." 

Taylor (1953) defines cloze as, "any single occuranoe 
of a successful attempt to reproduce accurately a part 
deleted from a •message* (any language produce) by deofdlng, 
from the^ context that remains, what the missing word should 

be* (p. ^16). , ... 

' Taylor (1953) further defines the ftlozn procedure ass 

a method of intercepting a message from, a 'transmitter 
' (writer *or^ «pe^teer )'-^ ^mutldat ing -its -language patterns 
by deleting parts, and so administering it to 
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receivers (readers and listeners) that their attempts 
to make the patterns whole again potentially yield a 
considerable nxmber of cloze units (p. ^H6). 
In other words, the cloze procedure Involves the completing 
of passages from which words have been systematically deleted, 

Taylor (1956, 1953) and Bormuth (I968) describe the 
cloze readability procedure. Basically, the procedure 
involves selecting passages Whose "difficulty is to be 

determined, deleting every fifth word In the passage, 
replacing the deleted words wlth»-blanks of equal length, 
duplicating the tests, administering the tests to students 
who have not read them before,' having students complete the 
blanks by filling In the word they think was deleted, and 
scoring the responses. The student has made a siicoessful 
cloze If he completes the^blank with the correct word. 

On the surface,' the cloze procedure may appear to be 
.the same as a sentence completion or "f lll-in-'the-blank" 
test; however. If one looks more critically, one will notice 
differences. In sentence completion tests, the words are 
omitted after being pre- evaluated. In oloze, words are 
deleted systematlcallyi there Is no pre-evaluatlon involved. 
In sentence completion, whole phrases or clauses may be 
deleted. In cloze, only one word Is deleted at a time. 

The cloze-procedure has been used in a variety of ways. 

5 
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These uses Include itst (a) correlation with standardized 
reading tests, (b) use as a measure of readahility , (c) use 
as a measure of verbal ability, (d) use as a measurement of 
foreign language achievement, (e) use as a teaching device, 
and (f ) use as a measure of listening ability. 

Literature iri which the cloze procedure has been used 
as a measurement of listening comprehension will be reviewed 
in this paper. Secondly, suggestions for further research 
will be offered. 

Taylor (1956) reports that the cloze procedure can be 
used to discriminate between the "listenabilities* (p. ^^7)^ 
of materials. Two radio news stories of abottt 170 words 
each were read in two ways by an experienced radio announcer. 
The first version was written and read in a manner that was 
easy to comprehend; the second, in a manner that was 
difficult to comprehend. (Both versions followed grammatical 
rules.) The deletions were fourteen spoken words apart. A 
buzzer was sounded for about four seconds and a serial 
number was read for the deleted words. Subjects had to find 
the serial number on their answer sheets and write down their 
guesses for the missing words. Taylor states that the cloze 
scores discriminated between the "listenabilities* of 
"good" and "bad" script; the good script being more listehabl 
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VeaTer (Dissertation Abstract, I96I) used oral and 
written presentations of cloze with two groups of Juniors 
and seniors (80 subjects In each) at the University of 
Georgia to determine if there was a difference in the 
ability of subjects to coaplete the cloze. One group 
received reading tests and was allowed as tnany repetitions 
of the aural cloze as desired, The other group received the 
reading cloze and was allowed' only one repetition of the 
aural cloze selection. Using a 2x2x4 analysis of variance. 
Weaver concluded that students performed significantly better 
in "any-word* silent cloze than aural, but that lexical 
deletion of nouns and main verbs was completed equally well 
by listening end silent reading. 

Weaver and Kingston (I963) studied the relationship of 
cloze tests to standardized reading tests. One hundred 
sixty students in the Junior class at the University of 
Georgia took part in this study. Eight cloze tests were 
administered. These Included a structurisil deletion of 
essay materials read silently, a lexical deletion of essay 
materials read silently, a structural deletion of a speech 
read silently, a lexical deletion of a speech read silently, 
a lexical deletion of ecsay material listened to, a 
structxiral deletion of a speech lis-cened to, and a lexical 
deletion of a speech listened to. 



These eight cloze tests were presented » In random 
order 9 to groups of eight to twelve students* Each test 
contained forty cloze items and the exact word was required 
to Make a successful cloze. A thirty minute time limit was 
placed on the reading and listening cloze tests. Students 
were allowed to hear the listening tape through completely 
and to ask the examiners to reread any sentences in which a 
cloze item appeared before completing the cloze. Pauses of 
four seconds were used to indicate a deletion. 

Other tests administered included thet Davis Reading 
Test, Modem Language Aptitude Test (KLAT) number learning 
subtest, MLAT phonetic script subtest, MLAT spelling subtest 
MLAT words in sentences subtest, MLAT paired associates sub- 
test. Sequential Tests of Educational Progress (STBP) 
listening subtest, Ohio State Psychological Examination 
(OSPE) vocabulary subtest, and OSPE reading comprehension 
subtest. 

The correlations between the lirtenlng cloze tests 
and other tests ranged from .15 to .69. The highest 
correlation was .69 in which a structural deletion of a 
speech-listening was correlated with a lexical deletion of 
essay material-reading, l^he point needs to be made that 
these are both cloze tests.. A correlation of .60 was found 
between lexical deletion of speech-listening and the Davis 

> 
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Heading Test. A correlation of .61 i^s found between 
structural deletion of speech-listening and the Ohio State 
Paychological Examination (reading comprehension). Other 
positlTe correlatinns were found between structural deletion 
of essay material-listening and lexical deletion of e^y^ 
material-reading; lexical deletion of a speech-reading and 
structviral deletion of essay materlal-llsten|.ng; and the 
STEP listening subtest and the listening cloze tests., 

(r=.^8, A5* .50, .5^). 

Jongsma (undated) wrote that the relatively low 
correlations "seem to suggest that the cloze listening tests 

were measuring factors other than listening comprehension" 

V 

(p. 21). 

Neville and Pugh (197n~compm-the- performs pf 
children on similar reading and listening cloze tests 
with regard to the number and types of errors made. Sixty- 
six pupils (40 girls, 26 boys) ages eight years ten months 
to ten years nine months, from lower and lower middle class 
homes, and attendiu^s a Roman Catholic primary school in 
Yorkshire, were subjects In th*^; experiment . The students 
were assigned randomly to two groupfs (X=32 subjects, Y«3^ 
subjects), forms R end B of the GAP Reading Test were used. 
(This is a cloze type test with approximately every tenth 
word deleted, and standardized in the %lted Kingdom). Both 
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forms of the GAP were recorded by 8 female adult reader 
Kith approximately ten second pauses for each deleted word. 
An answer paper with numbered lines for the deletions was 
prepared • 

The two groups first received the reading versions of 
the GAP Test following manual instructions. After a short 
rest, groups X and Y received the listening version. In 
scoring, synonyms were not allowed, and errors were classified 
as omissions, wholly inappropriate responses, or incorrect 
fesponses according to the manual* but linguistically 
appropriate according to context. 

The twot means of the total sample for reading and 
listening were compared using a t-test for related (Total 
Reading M=24.60, Total Listening K=13.87, t«5.93^ p<*00l)* 
A lower listening mean for the whole sample is in ^vidence. 
The listening mean average is equivalent to a ree^ding age of 
eight years nine months on the GAP, the reading mean of 
ten years seven months, and the mean chronological age was 
nine years nine months. The results also pointed to a very 
high frequency of omissions on both fonas of the listening 
version of the GAP. 

Using the Pearson product jnoment, significant correlations 
(r=.50,p< .01) were found between 'the NFEK Reading Test 
AD (which was administered one month before the investigation) 
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and the GAP Listening Test^ and between ttie GAP Reading 

and GAP Listening Tests (r».48, p •01). The Results appear 

to establish GAP as a listening comprehension test. 

Swalui (Dissertation Abstract, 1971) used the cloze 
procedtire (with a ten percent random deletion of lexical 
words) with 32h students In grades two, 1j.hree and four (108 
from each) to study the effect of testing materials that had 
readabilities equivalent to students'/grade placements upon 
oral and silent reading and listening comprehension. 

Using analysis of variance formulas and Tukey's 
Honestly Significant Difference Test (significance established 
at p<.05) oral reading was significantly better than silent 
reading or comprehension or listening comprehension at. the 
second grade level; no significant differences were found 
between methods at the third and fourth grade levels; silent 
reading of above-average students was significantly better 
than listening in grades two and four; and students with 
average reading ability tended to comprehend equally well 
with all three approaches (significance not found). Listening 
was significantly more effective for comprehension than silent 
reading for below-average students at all three grade levels, 
and significantly different from oral reading only in grade 
threes 

' Kennedy and Weener (1973) used the cloze procedm'e as a 
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tneans of, and to det^mine if,, training poor readers to 
attend auditorlally and visually to contextual cues would 
"improve listening and/or reading comprehension. More specif r 
ically, Kennedy and Weener hypothesized thati (l) experi- 
mental groups receiving visual and auditory training would 
perform significantly better than control groups on ligtening 
and reading tests;'and (2) students who received auditory 
tfalnlng would make the greatest gr.ins on listening- tests; 
and students who received visual training would make the 
greatest gains on reading co^iprehension tests. 

Eighty students, completing third grade, with reading 
grade levels between 1.9 and .3.5 (as determined by the 
Metropolitan Achievement Test) and reading between the 1-2 
(first grade, second half) and 3-1 (third grade, first half) 
on paragraphs fro^ the O^iStl^oy^tBaslc^^ ^^^^ 
the st^dy. The subjects were ranked from high to low 
according to test res-nlts, and ^.ssigned to one of four groups. 
The top four students were assigned randomly to one of the . 
four groaps, the next four were assigned randomly, et^i., 
•until all eighty students were assigned to a group (20 student 
in each group; two experimental groups and two control groups) 

The experimental listening group received five, 20- 
mlnute, training sessions (a total of one hour forty minutes) 
in replacing words deleted from sentences and stories (ten 
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Sentences and one story diirlng each session) which were 

presented audltorlally. A bell was rv&is each time a word 

/'' •■ 

was deleted, and subjects could have/sentences repeated. 
The experimental reading group received similar training t 
but the sentences and stories were presented visually. To 
account for the possible Hawthorne effect, the control 
reading group received individualized oral reading ir/struc- • 
tion for five, twenty minute, training isessiohs. The control 
class group remained in the regular classroom and received 
no special treatment . 

The cloze materials were designed with one word 
deleted from each sentence; and sin attempt Mas made to have 
words taken proportionally from the beginning, middle ^nd 
end of sentences. Also, prepositions and conjunctions, verbs 
and adverb's, adjectives and articles,, and noiins were taken 
out during' one sessjjon each. A successful cloze was made if 
' the word was "syntajtically or semantically correct" (p. 533)« 
Using a Least Significant Difference test it was: found 
thati (1) the experimental listening group scored higher 
on the Durrell Listening Reading Series listening compre- 
hension subtest than the reading comprehension subtest 
(p <'.10);^(2) the experimental reading group scored 
significantly higher (p<.05) on the reading comprehension ^ 
subtest of the DLB than on the listening comprehension subtest 
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(3) the experimental groups did , significantly better (p ^•05) 
than the control class on the listening comprehension 
subtest; '(^) the experimental groups performed significantly 
better (p <«05) than the two control groups on the Cloze 
Procedure subtests; and (5) there was no significant differ- 
ence between the experimental groups on. the Cloze Procedure' 
tests* 

CraTcer (Dissertation Abstractj, 1971) used the 
Clozentropy Pro'cedure to test the oral English ability of 100 
pre-llt^rate lower middle class Angle, Black, Navajo and 
Spanish children (25 in a group) of legal age for first grade* 
An Oral Clozentropy Test of Siglish Proficiency was developed, 
and a CDC 6^100 compiater program analysis of variance was 
conducted to compare group means. Ciraker concluded - that \ 
98 of the 100 students, understood and could use the procedure. 

Smith (Dissertation Abstract, 197^) studied the ability 
of S'l' kindergarten children to. use context on aural cloze. 
She Investigated which parts of speech (nouns, verbs, 
adjeottv.es, and adverbs) were easiest to supply, and the 
easiest sentence Dositlon for children to complete the cloze. 
Tests were administered Individually and orally to each 
student and exact and equally acceptable replacements were 
allowed In scoring. Using a one-way analysis of variance 
and multiple comparison procedures according to the Scheffe' 
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method, she concluded that students found 'noims in end 
position the easiest to complete and verbs in end position 

the next easiest. Noun deletions were found to be easiest 

' 

and verb, the next easiest of randoiuiel-et^iotts^"--''^e^ 
scores did not incxesrsST significantly as a result of 
accepting ecfually^ acceptable responses; and there was no sex 
difference in performance. ' / 

Kingston and Weaver (1970) attempted to determine/ the 
feasibility of using the cloze procedure as a teaching 
technique with ^sadvantaged rural children, and to determine 
its predictive poweiT* with achievement tests at the first 
\ grade level. One hundred and eighty- two rural, white 
\ dlsfi^dvantaged first grade 'students were taught the cloze 
/.procedure. , Approximately half of the students were used to 
; obtain reliability data for the instruments, eight were 
! dropped from the study because they were absent during testing 
or left school; data from Ik students 'was used for the 
' regression analysis which was calculated. 

A planned strategy was used to teach students how to use 
.the cloz<5 procedure. Using the language experience approach, 
ithe- children made up stories; and the teacher wrote these on 
the boavd. The teacher then omitted a lexical item and the 
children supplied words they thought could complete the 
Ipassage and -still make sense, ^he' next step Involved typing 



stories the students developed and presenting them to the 
students at a later time* Words were then deleted from these 
stories* At a more advanced level, students were shown books 
which had rebus pictures* The number of pictures present^ 
was decreased gradually and blanks were used in place of thenr. 
Students supplied the missing words* In the next stage, the 
teacher read aloud stores which the students had written ^ 
and the students^ followed along* Deleted versions of the 
stories were then read aloud by the teacher and students 
supplied the i^lssing words* At first only two to three words 
were deleted per 100-200 word stories* Aft a final step, in 
the^cloze procedure training, students were given stories - 
they had not written in class, and asked to fill-in the 
missing words* Words which nad the first letter correct in 
their spellings were counted as connect. 

The Lee-Clark Readiness Test was administered during the 
week preceeding, and also during the first woek of school* 
The Glnn Fre-?rlmer test wa6 administered approximately mid 
way through the year* Cloze tests were administered during 
February and March, Gir>ft Primer Test was administered 

dtiring April, and the California Achlevjament Tests dxirihg 
Nay. 

Pour types of clozfe tests were administered. The first; 
type was an "any-word* cloze. With this test, students 



16 



15 

read the passage containing all the words before completing 

the cloze test. Every fifth or tenth word was deleted on a 

50 items cloze test which was administered in five sessions 

with ten items each. The test-retest reliability for this 

test was .78. The second cloze test was a ••multiple-choice, 

structural cloze. Each deleted word was paired with a 

distractor of the same grammatJcal class. After five 

deletions, ten words were listed across the page from which . 

the student could select choices. The test-retest reliftbllltjt 

was' .91, The third type of cloze test was a *multiple- 

choice, lexical* cloze. A fifth word deletion was carried 

out on nouns, verbs arid adjectives. Each word was paired 

with a distractor of the same grammatical class. After five 

deletions, ten words were listed across the page. The 

test-retest reliability was .76* ^he last type qt cloze 

used was an "aural r«ding" cloze. This is the. same as the 
. ■ ■ ■ % 

*ariy-wo3rd* cloze, except that the teacher reads the pa8sag|6(,j 

aloud as the student reads it silently. The teacher^stops ' 
for thirty seconds at each cloze blank. The test-retest 
reliability was ,85, 

A step-wise regression was computed on the California 
Reading Test using the -cloze tests, Lee Clark Readiness, 
Ginn PreiPrimer and Ginn Priner Tests as independent variables 
(Sig, P at ,01 level for all). Aural-cloze correlation 
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coefficients with other testing measures were: California 
Reading .62; California Arithmetic .66; California Language 
.57; Multiple-choice, Structural Cloze .8.7't Multiple-choice, 
Lexical Cloze 173; and Straight Cl^o'ze ,.76. 

Kingston and Weaver's best conclusion appears to be 
th&t first grade children were able to handle oral and 
written cloze. They also concluded that the cloze, tasks 
were better predic6\tTs of standard reading scores obtained 
at the end of the- year than were the readiness and basal ■ 
reader instruments. 

Little research has been conducted using the atiral 
cloze procedure as a measure of listening comprehension. 
Nine studies have been cited in this paper. Prom these 
studies, the following conclusions appear to' be. In order. 
^ Aural cloze may be used as a means of discriminating 
She listehabillties of materials. Taylor (1956) concluded 
that cloze scores discriminated between the "listenabilitles 
of •^good" and "bad* script,— the good script being more 
listenat>le. 

.' ^- 

-Aural cloze may be used as a measure b.f listening 
comprehension. Weaver (I96I) determined that nouns and 
main verbs could be fllled-ln equally well by silent 
reading or 'listening. Weaver and Kingston ■ (I963) found a 

y 



significant correlation {p<.05) between aural and reading 
cloze passages, aural cloze tests and standardized reading 
comprehension tests, and aural cloze tests and the STEP 
listening test, Nevlll and Pugh (1971) ^ound significant # 
correlations between the NPER Reading Test and the GAP Llstenliig 
Test (p-^.Ol) and between the GiiP Reading Test and the listen- 
ing version (p<e01). Swain (1971) found that studeh^ts with . 
average reading ability comprehended grade level oral and 
silent reading and listening material equally well when -the 
cloze procedure was used as a testing instrument, Kennedy 
and Weener (1973) fdund that students trained in the cloze 
procedure scored significantly better (p <.05) on the ' 
I>arrell Listening-Reading (listening comprehension subtest) ^ 
than the control group; and there were no significant 
differences between the .experimental groups on the Cloze 
Procedure Tests, 

Notm deletions appear to be the easiest to supply in- 
aural cloze. Smith (197^^) concluded that noun deletions were 
the easiest to supply, followed by main verbs. 

Aural cloze may be used with students in first grade. 
Cracker (1971) concluded that children of first grade age., 
could understand- and perform on aural cloze tests, Kingston . 
and Weaver (1970) concluded that first grade children were___ 
able to handXe-o^ral^and written cloze. Arid Smith (197^) 
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successfully used kindergarten students in a study of the : ^ 
use of context on atxral cloze parssages* 

. f 

There is a tendency for researchers to attempt to 
ascertain the relationship between aural (listening) cloze 
tests and measvires of silent reading rather than listening 
comprehension. Weaver and Kingston {I963) correlated] cloze 
tests with the Davis Reading Test, Modern Language Aptitude 
Test and the Ohio State Psychological Examination. Kingston 
and Weaver (1970) correlated aural cloze tests with the^ 
California Reading Test,- Lee Clark Readiness, Glnn Pre-Prlmer, 
and Ginn Primer Tests. Neville and Pugh (1971) correlated 
v-the reading form of the GAP Reading Test with a listening 
version of the GAP and the NPER Reading Test. 

Further Investigation Into the relationship between 
aural cloze tests and standardized listening tests such as 
the Durrell-Llstenlng Reading: 'Listening Subtest, Stanford 
Achievement I Listening Comprehension Subtest, Assessment of 
Children's Language Comprehension, Brcwn-Carleon Listening 
Comprehension Test, ^Jooperatlve' Primary Tests: Listening, 
Orr Graham Listening Test, Progressive Achievement Tests of 
Listening Comprehension, Sequential Tests of Educational 
Progress: Listening, and Tests for Auditory Comprehension: of 
Language appear .in order. 
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Investigation into the relationship between aural cloze 
tests and the listening comprehension subtests of reading 
inventories such as the Denver Public Schools Reading 
Inventory., Classroom Reading Inventory by Sllvaroli, 
^Diagnofetlc Heading Scales by Spache, Standard Reading 
Inventory by McCracken, and the Individual Reading Plaoement 
Inventory (Pollett Educational Corporation) appear in o^der.^^ 

Another area to investigate is whether sex differences 
exist in aural cloze results. Smith (197^) found that, 
"there is no evidence to Indicate that there are sex 
differences for. ..aural cloze tasks" (p. 35*)0-A) in her 
investigations with kindergarten children. Investigations 
at other grade levels are in order. 

A fourth area to Investigate is the predi-ctive power 
of aural cloze in assessing reading readiness-. Craker (1971) 
used an orally administered cloze test vrith children cf 
first grade age. Smith (197^) showed that kindergarten 
children could use aural cloze. Kingston and Weaver (1970) .- 
showed that* first grade children could use aur«^l cloze. 
They also attempted to d'etermine its predictive power over a 
readiness test with achievement' tests, but administered the 
readiness test at the beginning of the school year and, didn't 
administer the aural cloze tests until February and March,' 
well into the school year. 
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A review of evailable literature on aural cloze and 
a number of areas which are in need of further investigation 
have been presented ir. this paper^ It should be apparent 
that there is much to investigate in the use of atiral cloze. 
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